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Abstract 

Research and services continue to expand to community-based programs serving individuals diagnosed 
with autism. A focus of great interest in those efforts is that of language acquisition and functional usage. For the 
purpose of this evaluation, language acquisition interventions are grouped into two broad categories, contrived and 
naturalistic. Contrived interventions include those in tightly controlled settings with specific instructional sequences 
(e.g. discrete trials), whereas naturalistic interventions are defined as less structured and with instructional sequences 
that change with the child’s interest. This meta-analysis focuses on interventions of spoken language developed for 
children diagnosed with autism. Contrived teaching approaches were compared to naturalistic approaches during 
intervention, generalization, and maintenance conditions by calculating the percentage of non-overlapping data 
(PND). The results illustrate that naturalistic interventions are more effective than contrived interventions for 
teaching language skills and for maintaining the skills learned. The data also indicate that contrived approaches 
result in greater generalization than naturalistic approaches. Implications and future directions for the acquisition 
and generalization of acquired language skills and the importance of treatment integrity are discussed. 

Keywords: autism, contrived teaching, discrete trial teaching, DTT, language interventions, meta-analysis, natural 
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Autism is not only of the most prevalent developmental disabilities, but it is also the 
fastest growing according to the Autism Society of America, (2008). The Center for Disease 
Control and Prevention statistics indicate that 1 in 150 8-year-old children in the United States 
have an autism spectrum disorder (CDC; 2007). The rising incidence may be due to increased 
awareness, early identification markers and screenings, and more sensitive and specific 
assessment diagnostic instruments. As the number of children with the disorder rises, so too 
does the need for qualified therapists and effective interventions to maximize each child’s full 
potential. As such, the CDC recommends that individuals diagnosed with autism receive 
evidence-based, early intervention services as soon as possible (CDC; 2007). 

According to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition 
(DSM-IV; 1994), autism spectrum disorder (ASD) is described by significant deficits in three 
behavioral domains: 1) qualitative impairment in social interaction, 2) qualitative impairment in 
communication, and 3) restricted, repetitive and stereotyped patterns of behavior, interests, and 
activities. Communication deficits include a delay in the development of spoken language. 

When language does develop, impairments in conversational language occur with high frequency 
(DSM-IV; 1994). Many instructional models (e.g. discrete trial teaching, incidental teaching, 
pivotal response training) specifically teach spoken language as part of the instructional sequence 
to remediate these debilitating communication deficits. 

As stated above, there are multiple approaches used to teach individuals with 
developmental disabilities spoken language just within the field of applied behavior analysis 
(ABA). Historically, the approach most associated with ABA is discrete trial teaching (DTT). 
Discrete trial teaching is a systematic and structured teaching methodology, consisting of 
“discrete” trials. A discrete trial consists of one concise instruction, a learned response, and a 
consequence highly controlled by an instructor. Discrete trial teaching sessions generally occur 
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at an isolated table in a designated area of a home or school and thus the model has received 
significant criticism over the years (Steege, Mace, Perry, & Longnecker, 2007). Therefore, for 
purpose of this investigation, all research using DTT is referred to as contrived approaches 
because the instructional strategy is not “typical” of a naturalistic setting. 

In addition to the setting and approach being contrived, discrete trial critics have argued 
that there is a lack of skill generalization, that the instructional approach only produces rote 
responding, and that there is an inability to teach sequential chains since instruction only occurs 
as discrete trials (Steege et ah, 2007; Sundberg & Partington, 1998). For these reasons, critics 
have referred to DTT as an analog training condition and not likely to generalize to natural 
contingencies of reinforcement. 

Autism interventions have also evolved to address some of the criticisms of DTT and to 
better address the needs of the population. Some researchers worked toward altering teaching 
strategies in hopes of achieving more promising outcomes. For instance, Koegel, O’Dell, and 
Koegel (1987) conducted a study in which they manipulated teaching variables to include more 
functional teaching stimuli, naturalistic reinforcers, and teaching within the natural environment. 
The results of this study suggest that these teaching methods resulted in greater generalization of 
language skills. In addition, other ABA approaches have emerged that specifically focus on 
training in the natural environment and are thus called, naturalistic approaches. Naturalistic 
approaches include incidental teaching (Hart & Risley, 1975, 1982), natural environment 
teaching (Sundberg & Partington, 1998), pivotal response training (Koegel, Koegel, Harrower, & 
Carter, 1999), and enhanced milieu teaching (Hancock & Kaiser, 2002). The term naturalistic is 
used because the stimuli, prompts, and reinforcer availability occur naturally in the student’s 
environment. Naturalistic programming is generally considered more child-centered than 
contrived approaches. For example, instructors will often follow the child’s lead in a play 
situation and look for occasions to provide instruction in language skills not yet secure in the 
student’s repertoire. Additionally, there is an assumption with this approach that the acquired 
skills will generalize more readily than contrived approaches because the reinforcement for 
correct responding is accessible in the natural environment (Cowan & Allen, 2007; Sundberg & 
Partington, 1998). Finally, proponents of naturalistic approaches argue that acquired language is 
less robotic and rote than language acquired through a contrived approach. 

There is little extant research evaluating the relative effectiveness of the approaches 
mentioned above. Additionally, a comprehensive search reveals no quantitative analyses 
comparing these approaches. Comparative reviews typically fall into two broad categories, 
literature reviews and meta-analyses. Literature reviews synthesize and summarize previous 
evaluations but offer no quantitative analyses. Whereas, a meta-analysis is a method of 
synthesizing, quantifying and summarizing the results from independently conducted 
evaluations. Glass (1976) argued that a meta-analysis is the statistical approach to review extant 
data that is superior to narrative reviews. He explained that meta-analysis is the “analysis of 
analyses.” 

Scruggs, Mastropieri, and Castro (1987) outlined a procedure for quantitatively 
evaluating the effectiveness of treatments for single-subject research. The percentage of non¬ 
overlapping data points (PND) is a non-parametric measure that examines the difference between 
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the baseline and treatment phases of individual studies. Kazdin (1978) suggested that evaluating 
non-overlapping data between baseline and intervention phases is a reliable method of evaluating 
treatment effectiveness. 

Scruggs, Mastropieri, Fomess, and Kavale (1988) conducted a meta-analysis using PND 
to assess early language interventions for children identified as handicapped. The investigation 
focused on the effects of language interventions on frequency of verbalizations. The researchers 
concluded that interventions which included specific generalization procedures produced more 
favorable outcomes than interventions involving the “train and hope” philosophy for increasing 
frequency of verbalizations for that population. 

Other researchers have used PND to evaluate other topics related to skill acquisition and 
programming for individuals diagnosed with autism. Bellini, Peters, Benner, and Hopf (2007) 
analyzed social skills interventions for children with autism in school settings. The authors 
concluded that social skills interventions in schools were minimally effective. Bellini, and 
Akullian (2007) compared video modeling and video self-modeling for the ASD population by 
computing PND. They concluded that video modeling and video self-modeling were both 
effective for social skills acquisition and maintenance. An evaluation of the Picture Exchange 
Communication System (PECS) literature reported effectiveness for mand initiations (Devis & 
Tincani, 2008). Using the percentage of non-overlapping data (PND) method, the authors 
reported that of the studies that were included, PECS proved to be effective in establishing 
mands (i.e., functional communication). 

The present meta-analysis evaluates the relative effectiveness of interventions for 
children with autism that lead to the acquisition of spoken language. The extant literature 
included in the investigation was categorized into the two broad approaches described above, 
contrived and naturalistic. Intervention, maintenance, and generalization effects were measured 
by calculating the percentage of non-overlapping data in order to answer three questions: a) do 
contrived approaches demonstrate a greater change from baseline to intervention as expected? b) 
do naturalistic approaches show comparatively greater generalization?, and c) do the two 
approaches differ comparatively over time? 


Method 


Search Criteria 

Relevant articles were searched using PsycINFO and Educational Resources Infonnation 
Center (ERIC) databases by using various combinations of the following search terms: analog, 
Asperger’s, autism, autoclitic acquisition, communication, contrived, discrete trial, discrete trial 
instruction, discrete trial teaching, DTI, DTT, echoic acquisition, incidental learning, incidental 
teaching, intraverbal acquisition, language, language acquisition, mand acquisition, milieu, 
natural, natural environment teaching, naturalistic teaching, FDD, pervasive developmental 
disorder, pivotal response, pivotal response training, structured teaching, tact acquisition, and 
verbal behavior. In addition, manual searches of the Journal of Applied Behavior Analysis 
(JAB A), and the Journal of Autism and Developmental Disorders were conducted in order to 
locate additional studies that were not captured by the original database searches. Finally, the 
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reference sections of the studies captured by the original searches were reviewed, and all 
research relevant to this meta-analysis were retrieved. 

Next, the first author ensured that the evaluations found from the searches described above 
met the focus of the study (i.e., inclusion/exclusion criteria) which was established a-priori. The 
inclusion/exclusion criteria was established to include only those studies that fell within the 
scope of the questions raised above, and to include: only research demonstrating experimental 
manipulations of language acquisition; only research using single-subject designs; multiple 
baseline, multiple probe, or ABA designs; research participants between the ages of 2 years and 
13 years in order to specifically focus on children; research participants that were diagnosed with 
ASD, Pervasive Developmental Disorder (PDD), or Pervasive Developmental Disorder Not 
Otherwise Specified (PDD-NOS); and finally each study had to present a graphic display of the 
data in order for PND values to be calculated (see Figure 1). 


Authors/Year 

Research 

Design 

Participants 

Number/Age/Diagnosis 

Dependent 

Measures 

Type of 
Intervention 

Phases Included 
in Analysis 

Krantz, 

Zalenski, Hall, 
Fenske,& 
McClannahan 
(1981) 

Multiple 

baseline 

4 children with autism 
(age 7-13) 

use of label, 
color, size, 
shape,& 
verb 

contrived 

Baseline 

Intervention 

Woods (1984) 

Multiple 

baseline 

2 children with autism 
(age 6) 

verbal 

tacting 

compare 
contrived & 
naturalistic 

Baseline 

Intervention 

McGee, 

Krantz, & 

McClannahan 

(1985) 

Multiple 

baseline 

3 children with autism 
(age 8-11) 

preposition 

use 

compare 
contrived & 
naturalistic 

Baseline 

Intervention 

Generalization 

Yamamoto, & 

Mochizuki 

(1988) 

Multiple 

baseline 

3 children with autism 
(age 10-11) 

"give me" 
and "that's 
not it" 
responses 

contrived 

Baseline 

Intervention 

Generalization 

Maintenance 

Miranda- 
Linne, & Melin 
(1992) 

Multiple 

baseline 

2 children with autism 
(age 10-12) 

use of color 
adjectives 

compare 
contrived & 
naturalistic 

Baseline 

Intervention 

Maintenance 

McGee, 

Almeida, 

Sulzer- 
Azaroff, & 
Feldman 
(1992) 

Multiple 

baseline 

3 children with autism 
(age 3-5) 

peer 

manding 

naturalistic 

Baseline 

Intervention 

Pierce, & 

Schreibman 

(1995) 

Multiple 

baseline 

2 children with autism 
(age 10) 

initiating and 
maintaining 
conversation 
with peers 

naturalistic 

Baseline 

Intervention 

Generalization 

Maintenance 

Taylor, & 

Harris (1995) 

Multiple 

baseline 

3 children with autism 
(age 5-9) 

mands for 
information 

compare 
contrived & 
naturalistic 

Baseline 

Intervention 
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Pierce, & 

Schreibman 

(1997) 

Multiple 

baseline 

2 children with autism 
(age 7-8) 

initiating and 
maintaining 
conversation 
with peers 

naturalistic 

Baseline 

Intervention 

Generalization 

Maintenance 

Koegel, 
Camarata, 
Koegel, Ben- 
Tall, & Smith 
(1998) 

ABA 

design* 

5 children with autism 
(age 3-7) 

intelligibility 
of various 
speech 
sounds 

compare 
contrived & 
naturalistic 

Baseline 

Intervention 

Buffington, 

Krantz, 

McClannahan, 

& Poulson 
(1998) 

Multiple 

baseline 

4 children with autism 
(age 4-6) 

pairing 

verbal 

responses 

with 

gestures 

contrived 

Baseline 

Intervention 

Generalization 

Finkel, & 
Williams 
(2001) 

multiple 

baseline 

1 child with autism (age 

6) 

intraverbal 

responses 

contrived 

Baseline 

Intervention 

Maintenance 

Sarokoff, 

Taylor, & 

Poulson 

(2001) 

multiple 

baseline 

2 children with autism 
(age 8-9) 

conversationa 

1 exchanges 

naturalistic 

Baseline 

Intervention 

Hancock, & 
Kaiser (2002) 

multiple 

baseline 

4 children with autism or 
PDD (age 2-4) 

conversationa 

1 utterances 

naturalistic 

Baseline 

Intervention 

Maintenance 

Jennett 

(2005) 

multiple 

probe 

6 children with autism or 
PDD-NOS (age 3-5) 

frequency of 
mands 

compare 
contrived & 
naturalistic 

Baseline 

Intervention 

Generalization 

Koegel, 

Carter, & 

Koegel 

(2003) 

multiple 

baseline 

2 children with autism 
(age 4-6) 

correct use of 

targeted 

morphemes 

naturalistic 

Baseline 

Intervention 

Generalization 

Esch, Carr, & 

Michael 

(2005) 

multiple 

baseline 

3 children with autism 
(age 6-8) 

echoic 

responses 

contrived 

Baseline 

Intervention 

Endicott, & 

Higbee 

(2007) 

multiple 

baseline 

4 children with autism 
(age 3-5) 

mands for 
information 

compare 
contrived & 
naturalistic 

Baseline 

Intervention 

Generalization 

Perez- 

Gonzalez, 

Garcia- 

Asenjo, 

Williams, & 

Carnerero 

(2007) 

multiple 

baseline 

2 children with autism 
(age 6-8) 

use of 

intraverbal 

antonyms 

contrived 

Baseline 

Intervention 

McGee, & 

Daly (2007) 

multiple 

baseline 

3 children with autism 
(age 4-5) 

use of social 
phrases 

naturalistic 

Baseline 

Intervention 

Generalization 

Naoi, 

Yokoyama, & 

Yamamoto 

(2007) 

multiple 

baseline 

3 children with autism 
(age 4-7) 

verbal tacting 

contrived 

Baseline 

Intervention 

Generalization 
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Jones, 

multiple 

2 children with autism or 

intraverbal 

contrived 

Baseline 

Feeley, & 

Takacs 

(2007) 

probe 

PDD-NOS (age 3) 

responses 


Intervention 

Generalization 


Figure 1 - Details pertaining to studies included in the meta-analysis. PDD = pervasive developmental disorder; 
PDD-NOS = pervasive developmental disorder-not otherwise specified. 

Some of the most well known discrete trial evaluations (Lovaas, 1987; McEachin, Smith, & 
Lovaas, 1993) did not meet the inclusion criteria for the meta-analysis. These studies were 
excluded largely because they used a group design, rather than a single-subject design. 

Additional studies were excluded if they didn’t specify the diagnosis of the research participants, 
didn’t provide graphic displays of data, didn’t involve measures of spoken language, or focused 
on research participants that were dually diagnosed. 

Categorization of Included Evaluations (Independent Measures) 

Each study was then classified as either “contrived”, “naturalistic” or both “contrived and 
naturalistic”. Contrived approaches (as described above) were operationally defined as highly 
structured teaching that consists of 1:1 instruction that occurred in an isolated location in a 
tightly controlled instructional environment. Studies using this approach involved the key 
components of a discrete trial as outlined by Koegel, Russo, and Rincover (1977). Those 
components include the presentation of a discriminative stimulus (Sd), a target response (or 
approximation), the programmatic and systematic delivery of a reinforcing consequence, and a 
specified inter-trial interval. In this approach, reinforcement for correct responding is not 
necessarily found in the student’s natural environment and the instructor controls access to 
reinforcers (praise, tangibles, etc.). In contrived approaches, the emphasis is usually placed on 
the topography of the response (i.e. what the child says), and not necessarily on the function of 
the response. 

“Naturalistic” teaching was operationally defined as a student-centered approach in 
which the instructor follows the student’s lead, and bases instructions on the student’s interests. 
Compared to contrived approaches, naturalistic approaches occur in a more natural setting (e.g., 
school, home, playground) with an array of play items available for the child to sample. The 
instructional arrangements are then “naturally occurring” in that the instructor waits for the 
student to engage in one of the play items, and then uses that item to shape or maintain verbal 
repertoires. For example, a naturalistic approach to teaching mands for information might 
include hiding a piece from a student’s favorite puzzle. Once the child asks, “where is the puzzle 
piece?” the instructor provides information leading to the missing item. The function, rather than 
topography, of language is heavily emphasized in naturalistic approaches. Pivotal Response 
Training (PRT), Natural Environment Teaching (NET), mand training, milieu teaching, and 
incidental teaching are all classified as “naturalistic” for the purposes of this study. 

Dependent Measures 

Examples of dependent variables for contrived approaches included labeling items such as 
color, size, shape, and verbs (Krantz, Zalenski, Hall, Fenske, & McClannahan, 1981), and 
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acquisition of intraverbal antonyms (Perez-Gonzalez, Garcia-Ansenjo, Williams, & Camerero, 
2007). For naturalistic approaches, dependent variables ranged from conversational utterances 
(Flancock & Kaiser, 2002) to peer manding (McGee, Almeida, Sulzer-Azaroff, & Feldman, 

1992). Some researchers compared both contrived and naturalistic approaches with the same 
dependent measures. Examples include the study by Woods (1984) which compared both 
approaches to evaluate the effects on verbal tacting, and the Endicott and Fligbee (2007) study 
which compared both approaches to evaluate the effects on mands for information (see Figure 1). 

Data Analysis: Percentage of Non-overlapping Data Points 

The PND values were calculated using the procedure outlined by Scruggs et al. (1987). 
That is, a ruler was horizontally placed at the highest baseline data point. Then, the number of 
data points in the treatment phase that exceeded that point were counted. The value of the 
number of data points in the treatment phase exceeding the highest baseline data point was then 
divided by the number of total data points in the treatment phase. That value was then multiplied 
by 100 in order to obtain a percentage (PND = number of treatment data points exceeding the 
highest baseline data point / total number of treatment data points * 100). For example, if three 
points in the treatment phase exceed the highest data point in the baseline, and there are four data 
points total in the treatment phase, the PND value is 75% (3/4 * 100 = 75%). Scruggs and 
Mastropieri (1998) establish cutoff criteria for evaluating quantitative data using PND: very 
effective treatments (over 90 PND), effective treatments (between 70 and 90 PND), questionable 
effectiveness (between 50 and 70 PND), and ineffective treatments (below 50 PND). 

Percentage of non-overlapping data scores were calculated from baseline to intervention 
for each research participant, across all dependent variables, in each of the 22 studies. 
Additionally, the PND scores were calculated for the generalization and maintenance (i.e., 
follow-up) conditions when applicable by comparing the appropriate condition (i.e. 
generalization or maintenance) to baseline conditions (see Figure 1). Research studies that 
compared both types of approaches were included in the mean PND by first categorizing the 
interventions in each study, then calculating the values from that intervention using the 
procedure described above. 

Inter-observer Agreement 

Two researchers independently evaluated each of the 22 studies to ensure that the 
inclusion/exclusion criteria were met. The researchers were in 100% agreement that all studies 
met criteria. Next, both researchers independently classified each experiment into one of three 
categories (based on the operational definitions above): 1) contrived programming, 2) 
naturalistic programming, or 3) both contrived and naturalistic programming. The researchers 
agreed 100% on the categorical grouping. 

Finally, PND inter-observer agreement (IOA) was calculated by dividing the number of 
agreements by the sum of agreements and disagreements, and multiplying that value by 100 
(agreements/agreements + disagreements * 100). IOA was calculated for all intervention, 
generalization, and maintenance effects. The IOA for PND calculations was 90.2%. 
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Results 


A total of 22 studies were found using the search keywords and then met the 
inclusion/exclusion criteria. Of the 22 studies, 21 were published in peer-reviewed journals 
between the years of 1981 and 2007. One doctoral dissertation met the inclusion criteria 
(Jennett, 2005). The 22 studies included an aggregate total of 65 research participants. The 
mean age of the research participants for the included studies was 6.2 years, with a range of 2 - 
13 years (see Figure 1). 

The majority of the studies included were multiple-baseline designs (n=19). Two studies 
used a variation of the multiple baseline design, described as multiple probe designs. One study 
used an ABA design. Ten studies were classified as contrived, six were classified as naturalistic, 
and six of the studies compared both types of interventions within the same experiment (Figure 
1 ). 


Quantitative Analysis 

A total of 103 PND scores were calculated, across participants and dependent variables, 
for contrived interventions from the 22 included studies (Figure 1). The PND scores for 
contrived interventions ranged from 0 - 100%, with a median of 90%. The mode for this 
distribution was 100% (see Figure 2). The mean PND score for contrived interventions was 65% 
(see Figure 3). 



Figure 2 - Median and Mode PND scores for contrived and naturalistic approaches to teaching language skills. 
Data were collected from the distribution of intervention, generalization, and maintenance effects. PND = Percent 
of Non-overlapping Data points. 
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Comparing Contrived and Naturalistic Approaches 
Mean Percentage of Non-overlapping Data Points 



Figure 3 - Mean PND scores for both contrived and naturalistic approaches to teaching language skills across 
intervention, generalization, and maintenance effects. PND = Percent of Non-overlapping Data. 


For naturalistic interventions, a total of 49 PND scores were calculated, across 
participants and dependent variables, from the 22 included studies (see Figure 1). The PND 
scores for naturalistic interventions ranged from 0 - 100%. The median for the distribution of 
PND scores for naturalistic interventions was 98%, and the mode was 100% (see Figure 2). The 
mean PND value for naturalistic interventions was 83% (see Figure 3). 

PND scores were also calculated to determine generalization effects for both contrived 
and naturalistic approaches to teaching spoken language skills. Seven of the 22 included studies 
involved graphic displays of generalization effects for contrived programming (see Figure 1). 
Within those seven studies, 35 PND scores were calculated across participants and dependent 
variables. The range of PND values for generalization for contrived approaches was 0 - 100%, 
with a median of 97%. The mode for this distribution was 100% (see Figure 2). The mean PND 
score for generalization for contrived teaching was 87% (see Figure 3). 

Six of the 22 included studies provided graphic displays of generalization results for 
naturalistic programming (see Figure 1). Of those six research studies, 30 PND scores were 
calculated across subjects and dependent variables. The range of PND scores for generalization 
of naturalistic approaches was 0 - 100%, with a median of 95%. The mode for this distribution 
was 100% (see Figure 2). The mean PND for generalization for naturalistic approaches was 72% 
(see Figure 3). 
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Finally, PND scores were calculated for maintenance effects for both contrived and 
naturalistic approaches to teaching spoken language skills. Only 3 of the 22 included studies 
provided measures of maintenance for contrived approaches (see Figure 1). Of those three 
studies, 11 PND scores were calculated across subjects and dependent variables. The range of 
PND scores for maintenance for contrived approaches was 0 - 100%, and the mode was 100%. 
The median for this distribution was 100% (see Figure 2). The mean PND for maintenance for 
contrived programming was 82% (see Figure 3). 

Four of the 22 included studies provided measures of maintenance for naturalistic 
approaches to teaching spoken language skills to children with autism (see Figure 1). Twenty 
PND scores were calculated from those four studies. The range of PND scores for maintenance 
for naturalistic approaches was 0 -100%. The median PND score was 100%, and the mode was 
100% (see Figure 2). The mean PND score for maintenance for naturalistic programming was 
88% (see Figure 3). 

Figure 3 compares the mean PND for intervention, generalization, and maintenance 
effects for contrived and naturalistic approaches to teaching spoken language skills. For 
intervention effects, contrived approaches scored a mean PND of 65%, while the mean PND for 
naturalistic language interventions was 83%. For generalization effects, the mean PND for 
contrived programming was 87%, and the mean PND for naturalistic programming was 72%. 

The mean PND score for maintenance for contrived approaches was 82%, compared to a mean 
PND of 88% for maintenance with naturalistic approaches. 

Discussion 

The results for this meta-analysis reveal that when comparing mean PND scores, 
naturalistic interventions were more effective than contrived interventions from baseline to 
intervention and baseline to follow-up conditions, but less effective when comparing the baseline 
to generalization conditions. These results were somewhat surprising. It was anticipated that 
contrived approaches would show the greatest change from baseline to intervention, but less 
change from baseline to generalization. That is, it was expected that the tightly controlled 
environment of the contrived settings would demonstrate a superior instructional arrangement in 
that the new skills would be acquired faster and with less variability than the naturalistic 
approaches. And, that the variability of the naturalistic approaches would promote the 
generalization of the newly acquired skills, after skill mastery was established, and therefore be 
apparent in the PND between baseline and generalization conditions comparatively. In other 
words, naturalistic approaches are designed to mimic conditions in which language is learned 
naturally, and therefore generalize to natural settings more readily. For example, the first time an 
infant babbles a sound that resembles “mama” in the presence of his/her mom, mom reacts 
favorably, and that sound is selected over others in the future. This natural contingency is 
deliberately mimicked in the naturalistic approaches. 

The generalization effects reported for the meta-analysis suggest that the ASD 
population generalizes spoken language skills more readily with contrived teaching approaches, 
rather than naturalistic approaches. The mean PND scores for both naturalistic and contrived 
generalization measures were considered effective by the scale provided by Scruggs and 
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Mastropieri (1998), although the mean for contrived generalization was 15% higher than the 
mean for naturalistic generalization. This finding is contrary to what the authors predicted at the 
onset of this analysis. It was anticipated that language skills learned through naturalistic 
approaches would readily generalize across settings, people, and/or stimuli, and that contrived 
approaches would not. The reason for this conjecture is that the instructions, stimuli, and 
reinforcers used in contrived interventions are highly specialized; thus, the skills learned through 
contrived programming would not likely generalize to natural contexts. 

One possibility for the unexpected results for generalization effects found in the meta¬ 
analysis is that a bias exists in the studies that included measures of generalization for contrived 
teaching. The fact that researchers were testing for generalization effects could imply that 
programming included specific features conducive to generalization (e.g. fading contrived 
reinforcers for naturalistic ones; including responding in other sehings in mastery criteria). At 
the very least, the researchers recognized the importance of generalization; hence, generalization 
effects were being measured. For example, Jones and Feeley (2007) focused on teaching 
spontaneous responses to verbal and non-verbal cues. The authors clearly describe the 
intervention approaches as discrete trial teaching. However, the generalization procedures 
consisted of fading edibles (i.e., contrived reinforcers) for more naturalistic ones, such as the 
teacher speaking louder in response to the student saying “what?”. Generalization probes were 
done in a novel sehing with novel teachers, and correction procedures were not used. The 
measures taken for generalization purposes were conducive to generalization of spoken language 
skills, and all contrived teaching should include such measures. In fact, the generalization 
procedures described in Jones et al. (2007) closely resemble naturalistic teaching, and the 
outcome for the research participants was favorable. Another possibility for these surprising 
results is that the repetition that is characteristic of contrived approaches is more conducive to 
generalization. In naturalistic teaching methods, the instructor often follows the student’s lead 
and capitalizes on any teaching opportunities. It is unlikely that there are many repetitive 
opportunities to practice target skills in naturalistic approaches. However, repetition is intrinsic 
to contrived teaching approaches. 

Although contrived approaches scored a higher mean PND for generalization, naturalistic 
approaches scored a higher mean PND for maintenance effects. Maintenance measures involve 
the occurrence of a response over time (Alberto & Troutman, 2006), which is a critical factor 
when teaching spoken language skills. Children with autism have a tendency to lose language 
skills over time (Goldberg, Osann, Filipek, Laulhere, Jarvis, Modhal, Flodman, & Spence, 2003). 
Therefore, research involving acquisition of spoken language should include maintenance 
conditions and measures of skill maintenance. 

A possibility for stronger maintenance effects for naturalistic programming is that the 
naturally occurring contingencies used during intervention phases continue to be available within 
the environment over time. The reinforcers in a contrived teaching approach would be difficult 
to sustain over time, which may lead to lower rates of the target behavior. However, for this 
analysis, the mean PND for maintenance effects for both types of programming were considered 
effective according to the scale provided by Scruggs and Mastropieri (1998). The difference 
between the two means was 5.7%. 
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Future Directions 

Children with autism do not readily learn spoken language without a specific 
intervention. Today, many interventions for children with autism begin with contrived teaching 
approaches during acquisition. Clinicians are more likely to move to a more naturalistic 
approach to train for generalization of language skills. However, the results of the current meta¬ 
analysis suggest this method is not optimal for teaching spoken language skills. These findings 
suggest that perhaps it would be more effective to devise language interventions utilizing 
naturalistic teaching approaches for acquisition with a progression to more contrived approaches 
to train for generalization. At the least, further investigation is warranted in order to examine the 
differences between naturalistic and contrived interventions for learners with autism. It may also 
be useful to examine different variables (i.e. attention span, stereotypical behaviors, number of 
established reinforcers, etc.) for this population in order to determine the most appropriate 
language interventions for individuals. Ultimately, clinicians and practitioners should use 
research in this area to develop an assessment tool to determine which teaching strategies are 
most beneficial for individual learners. 

Another direction for future research is to investigate to what extent researchers are 
actually measuring generalization in studies involving spoken language acquisition. For 
instance, can the child respond to various instructions? Or, are the discriminative stimuli for the 
target response a set of highly specialized instructions? Another area of interest is for 
researchers to address the functional nature of spoken language learned through contrived 
teaching. When a child leams to say “cookie” after being asked, “what do you want?” in a 
contrived approach, is the word “cookie” functioning as a mand with a specific reinforcer (i.e., 
access to the cookie)? Or is the word “cookie” functioning as a learned intraverbal response? 
Another aspect of examining the function of spoken language learned through contrived 
programming is to address the criticisms that rote, robotic language emerges. 

An additional direction for future research is to consider the intervention, generalization, 
and maintenance effects of various skills taught through contrived and naturalistic approaches. 
For instance, it would be interesting to examine if skills such as eye contact or imitation are more 
readily learned through naturalistic programming. It is possible that the best approach varies 
depending on the skill being taught. 

Although the findings of this meta-analysis are important to consider when devising a 
language intervention for a child with autism, one must consider that outcomes can be 
idiosyncratic. That is, each child benefits the most from an individualized approach, based on 
the principles of applied behavior analysis. It is possible that a comprehensive approach is best, 
including aspects of both naturalistic and contrived teaching approaches (see Sundberg & 
Partington, 1999). However, each intervention should consist of continuous assessment, 
empirically validated teaching methods, and data collection. These interventions should also 
heavily emphasize generalization and maintenance of skills learned. As with any teaching 
strategy, the skill level of the practitioners is of upmost importance, as well as treatment 
integrity. 
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Limitations: Analysis Methods 

A standard outcome metric for analyzing single-subject research has not been identified. 
PND has been criticized in the literature for not representing treatment outcomes accurately 
(Allison & Gorman, 1994; Levin, 1992). However, as Scruggs and Mastropieri (1987) explain, 
PND can be easily computed through graphic displays of data, and it is an evaluative method that 
provides a measure of treatment effectiveness in most cases. PND also allows for a greater 
number of studies to be included in the analysis than parametric approaches because there aren’t 
constraints regarding normality, homogeneity, and independence of single-case data. Scruggs et 
al. (1987) remind the reader that there is not one single approach for evaluating data of single¬ 
subject designs, just as there is not one approach for evaluating research involving group designs. 
PND measures are valuable because they can be easily interpreted by the reader. For the 
purposes of this meta-analysis, PND was the most appropriate metric for evaluating treatment 
effectiveness. 
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